Method and device for identifying wrist, method for identifying gesture, electronic equipment and computer-readable storage medium

ABSTRACT

The present disclosure provides a method and a device for identifying a wrist, a method for identifying a gesture, electronic equipment and a computer-readable storage medium. The method includes: obtaining a first image, the first image including a hand and the wrist; binarizing the first image to obtain a binary image; extracting a partial image from the binary image, where the partial image is obtained by removing at least of finger information from the binary image; identifying a principal direction of the binary image based on the partial image, and determining a target direction perpendicular to the principal direction; and determining a target position in the binary image where the binary image matches a first wrist feature as a wrist position in the binary image according to the target direction.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims priority to Chinese Patent Application No.201810391614.5 filed on Apr. 27, 2018, which is incorporated herein byreference in its entirety.

TECHNICAL FIELD

The present disclosure relates to the field of image processingtechnologies, in particular to a method and a device for identifying awrist, a method for identifying a gesture, electronic equipment and acomputer-readable storage medium.

BACKGROUND

With the development of science and technology, gesture identificationis increasingly required in application scenarios, so as to performcorresponding operations according to a result of the gestureidentification. The gesture identification technology in the related artcannot distinguish a gesture from an arm in an image effectively, andthe arm in the image may interfere with the gesture identification,thereby reducing the accuracy of the gesture identification.

SUMMARY

In a first aspect, embodiments of the present disclosure provide amethod for identifying a wrist, which includes:

obtaining a first image, the first image including a hand and the wrist;

binarizing the first image to obtain a binary image;

extracting a partial image from the binary image, where the partialimage is obtained by removing at least of finger information from thebinary image;

identifying a principal direction of the binary image based on thepartial image, and determining a target direction perpendicular to theprincipal direction; and

determining a target position in the binary image where the binary imagematches a first wrist feature as a wrist position in the binary imageaccording to the target direction.

In a second aspect, the embodiments of the present disclosure furtherprovide a method for identifying a gesture, which includes:

determining the wrist position using the method for identifying a wristaccording to the first aspect;

segmenting the binary image along the wrist position to obtain a gestureimage; and

identifying the gesture on the gesture image.

In a third aspect, the embodiments of the present disclosure furtherprovide a device for identifying a wrist. The device includes a memory,a processor and a program that is stored on the memory and executable bythe processor. When the program is executed by the processor, theprocessor is configured to:

obtain a first image, the first image including a hand and a wrist;

binarize the first image to obtain a binary image;

extract a partial image of the binary image, where the partial image isobtained by removing at least of finger information from the binaryimage;

identify a principal direction of the binary image based on the partialimage, and determine a target direction perpendicular to the principaldirection; and

determine a target position in the binary image where the binary imagematches a first wrist feature as a wrist position in the binary imageaccording to the target direction.

In a fourth aspect, the embodiments of the present disclosure furtherprovide electronic equipment, including a memory, a processor and aprogram that is stored on the memory and executable by the processor.When the program is executed by the processor, the processor isconfigured to:

determining the wrist position using the method for identifying a wristaccording to the first aspect;

segmenting the binary image along the wrist position to obtain a gestureimage; and

identifying the gesture on the gesture image.

In a fifth aspect, the embodiments of the present disclosure furtherprovide a computer-readable storage medium storing a program. When theprogram is executed by a processor, the method for identifying a wristaccording to the embodiments of the present disclosure is implemented.

In a sixth aspect, the embodiments of the present disclosure furtherprovide a computer-readable storage medium storing a program. When theprogram is executed by the processor, the method for identifying agesture according to the embodiments of the present disclosure isimplemented.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a flow chart of a wrist identification method according to atleast one embodiment of the present disclosure;

FIG. 2 is a schematic diagram of a binary image according to at leastone embodiment of the present disclosure;

FIG. 3 is a schematic diagram of a gesture image according to at leastone embodiment of the present disclosure;

FIG. 4 is a flow chart of a wrist identification method according to atleast one embodiment of the present disclosure;

FIG. 5 is schematic diagram of a gray projection curve according to atleast one embodiment of the present disclosure;

FIG. 6 is schematic diagram of a gray projection curve and a derivativecurve according to at least one embodiment of the present disclosure;

FIG. 7 is a schematic diagram of a partial image of the binary imageaccording to at least one embodiment of the present disclosure;

FIG. 8 is a schematic diagram of a principal direction of the partialimage according to at least one embodiment of the present disclosure;

FIG. 9 is a flow chart of a gesture identification method according toat least one embodiment of the present disclosure;

FIG. 10 is a structural schematic diagram of a wrist identificationdevice according to at least one embodiment of the present disclosure;

FIG. 11 is a structural schematic diagram of a wrist identificationdevice according to at least one embodiment of the present disclosure;

FIG. 12 is a structural schematic diagram of a wrist identificationdevice according to at least one embodiment of the present disclosure;

FIG. 13 is a structural schematic diagram of a wrist identificationdevice according to at least one embodiment of the present disclosure;

FIG. 14 is a structural schematic diagram of a wrist identificationdevice according to at least one embodiment of the present disclosure;

FIG. 15 is a structural schematic diagram of a gesture identificationdevice according to at least one embodiment of the present disclosure;

FIG. 16 is a structural schematic diagram of electronic equipmentaccording to at least one embodiment of the present disclosure; and

FIG. 17 is a structural schematic diagram of electronic equipmentaccording to at least one embodiment of the present disclosure.

DETAILED DESCRIPTION

In order to make the technical problem to be solved, the technicalsolutions and the advantages of the present disclosure more apparent,the present disclosure will be described hereinafter in detail inconjunction with the drawings and embodiments.

FIG. 1 is a flow chart of a wrist identification method according to atleast one embodiment of the present disclosure. As shown in FIG. 1, themethod includes the following steps:

step 101: obtaining a first image, the first image including a hand andthe wrist;

step 102: binarizing the first image to obtain a binary image;

step 103: extracting a partial image from the binary image, where thepartial image is obtained by removing at least of finger informationfrom the binary image;

step 104: identifying a principal direction of the binary image based onthe partial image, and determining a target direction perpendicular tothe principal direction; and step 105: determining a target position inthe binary image where the binary image matches a first wrist feature asa wrist position in the binary image according to the target direction.

The above-mentioned first image may be obtained by collecting an imageof a human hand through an image collecting device, such as a camera. Inthe embodiment of the present disclosure, when the obtained first imageis binarized, the color first image obtained by the image collectingdevice may be directly binarized. The first image contains imageinformation such as fingers, a palm and a wrist.

It should be noted that the wrist identification method according to theembodiment of the present disclosure may be applied to any electronicequipment capable of identifying images, such as a mobile phone, atablet PC, a computer, a game device, an intelligent control device or aTV, which is not limited herein. In addition, the electronic equipmentapplying the method may or may not include an image collecting device.If included, the image collecting device may collect the above-mentionedfirst image; if not included, the above-mentioned first image may becollected by other electronic equipment and then sent to the electronicequipment.

In the step of binarizing the first image to obtain a binary image, agray value of a pixel point on the above-mentioned first image may beconverted into 0 or 255, i.e., the entire first image is converted intoa monochrome image. FIG. 2 is a schematic diagram of a binary image ofthe hand according to at least one embodiment of the present disclosure.In FIG. 2, the first image is a binary image obtained by binarizing thecolor first image of the hand using the binarizing method based on askin color model. As shown in FIG. 2, besides the gesture image, thebinary image further includes part of an arm image. As shown in FIG. 3,the gesture image may start from the wrist and include the palm andfingers. The arm in FIG. 2 may disturb the identification of the gestureimage. In the practical application, the first image may be segmentedinto the gesture image and the arm image firstly, and then the gestureidentification is performed according to the segmented gesture image,which may avoid the interference of the arm, and improve the accuracy ofgesture identification. The wrist exists between the gesture image andthe arm image, so an effective wrist identification method is the key ofimproving gesture identification.

Usually, the collected first images may have different visual angles,which may affect the segmentation of the gesture image and the armimage. Therefore, it needs to determine an overall orientation of thefirst image, i.e., an overall direction of the above-mentioned binaryimage, referred to as the principal direction. The principal directionis also an extension direction from the arm to the wrist and palm. Inaddition, the above-mentioned binary image may be symmetrical about theabove-mentioned principal direction approximately. In the embodiment ofthe present disclosure, the detection mode may be to extract a handcontour in the binary image, and determine the principal direction ofthe hand contour, which refers to the following description of theembodiment, and is not limited herein certainly. For example, a middleline of the binary image may be taken as the above-mentioned principaldirection, but the accuracy of this method is lower than the accuracy ofidentifying the principal direction by detecting a line segment of thecontour of the binary image.

In the embodiment of the present disclosure, the preset wrist featuremay be defined in advance. For example, the above-mentioned preset wristfeature is that a width of the contour changes the most violently. Sincethe above-mentioned target direction is perpendicular to theabove-mentioned principal direction, the width of the contour the binaryimage may be understood as its length in the target direction. In thisway, the target position where the width changes the most violently maybe determined by the above-mentioned target direction. For example, fromFIGS. 2, 3 and 5, it may be determined that the human hand ischaracterized by extending from the arm to the palm, from the arm to thewrist, with a little change in the width of the human hand, and anobvious change in the width at the wrist position, i.e., the positionwhich is perpendicular to the above-mentioned principal direction andhas a violent change in width of the contour. Certainly, theabove-mentioned width changing the most violently is merely an example.For example, the above-mentioned preset wrist feature may also bechanging gently and then suddenly towards two sides. The two sidesherein are those of the above-mentioned target direction, therebydetermining the target position changing gently and then suddenlytowards the two sides in the binary image by the above-mentioned targetdirection. For example, from FIG. 2, it may be determined that the wristposition is the one which changes gently and then suddenly towards twosides.

The above-mentioned preset wrist feature may further be an end point ofa line segment parallel with the above-mentioned principal direction. Inthis way, the line segment of the contour in the binary image may bedetected, and the line segment of an arm edge parallel with theabove-mentioned principal direction may be selected, thereby taking theend point of the line segment of the arm edge and the position parallelwith the above-mentioned target direction as the wrist position, inwhich the end point is close to the palm direction.

It should be noted that the wrist identification method according to theembodiment of the present disclosure may be applied to the gestureidentification method based on image processing, for example, to agesture identification system of a classifier based on template matchingor a neural network. In this way, before the gesture feature iscalculated, the wrist identification method according to the embodimentof the present disclosure accurately identifies the wrist position,thereby improving the accuracy of gesture identification.

In the embodiment of the present disclosure, a first image is obtained;the first image is binarized, so as to obtain a binary image; an partialimage in the binary image is extracted; a principal direction of thebinary image is identified based on the partial image, and a targetdirection perpendicular to the principal direction is determined; atarget position in the binary image where the binary image matches afirst wrist feature as a wrist position in the binary image according tothe target direction. In this way, the wrist position may be determinedeffectively, thereby improving the accuracy of gesture identification.

FIG. 4 is a flow chart of a wrist identification method according to atleast one embodiment of the present disclosure. As shown in FIG. 4, themethod includes the following steps:

step 401: obtaining a first image, the first image including a hand andthe wrist;

step 402: binarizing the first image to obtain a binary image;

step 403: extracting a partial image of the binary image, where thepartial image is obtained by removing at least of finger informationfrom the binary image;

step 404: identifying a principal direction of the binary image based onthe partial image, and determining a target direction perpendicular tothe principal direction;

405: calculating a gray projection curve of the binary image or a grayprojection curve of the partial image in the target direction; and

406: determining the target position in the binary image as the wristposition in the binary image according to the gray projection curve ofthe binary image or the gray projection curve of the partial image,where a projection position of the target position in the targetdirection is a position where a curve slope changes the most rapidly inthe gray projection curve.

In the step of calculating the gray projection curve of the binary imageor the partial image in the target direction, the binary image or thepartial image is rotated. For example, the binary image or the partialimage is rotated by a target angle which is an included angle betweenthe principal direction and a preset reference direction, which maychange the principal direction of the rotated binary image or thepartial image to be the preset reference direction, thereby calculatingthe gray projection curve of the binary image or the partial image inthe target direction. The above-mentioned preset reference direction maybe a vertical direction or a horizontal direction. Certainly, rotationmay not be performed. For example, the gray projection curve of thebinary image or the partial image in the target direction may becalculated directly. By rotation, the gray projection curve may take thehorizontal direction or the vertical direction as a reference, which ismore readable by persons skilled in the art. For example, as shown inFIG. 5, an abscissa is parallel with the horizontal direction, andrepresents that the gray projection curve extends in the principaldirection; an ordinate is parallel with the vertical direction, andrepresents numerical values on the gray projection curve, which in thisway may make the abscissa of the gray projection curve parallel with thehorizontal direction, more readable by persons skilled in the art.

In the embodiment of the present disclosure, it is assumed that a pixelmatrix of the binary image has a size of A*B. As shown in FIG. 5, theprincipal direction of the hand is a row direction of the image, and thetarget direction is a column direction of the image. In case ofcalculating the gray projection curve, a number of each row in the pixelmatrix is the abscissa x; the ordinate y of this row is obtained bycalculating a sum of gray values of B pixels of each row in the binaryimage matrix. In this way, A (x, y) coordinates are connected into acurve, so as to obtain the gray projection curve in the columndirection, i.e., the gray projection curve in the target direction.After the above-mentioned gray projection curve is calculated, theposition where the slope changes the most rapidly in the gray projectioncurve may be determined, and understood as the position where the changeis the most violent. The human hand is characterized by extending fromthe arm to the palm, from the arm to the wrist, with a little change inthe width of the human hand, and an obvious change in the width at thewrist position, i.e., the projection position of the wrist of the binaryimage in the target direction is the position where the slope in thegray projection curve changes the most rapidly.

By the steps 405 and 406, the target position corresponding to theposition where the slope in the gray projection curve changes the mostrapidly in the binary image may be determined as the wrist position,thereby improving the accuracy of wrist identification.

It should be noted that in the present embodiment, the gray projectioncurve of the binary image or the partial image in the above-mentionedtarget direction may be calculated. However, since the partial image isa partial image in the binary image, after the above-mentioned targetposition of the partial image is determined, the target position in theabove-mentioned binary image may be directly determined because the twopositions are the same. In addition, since the gray projection curve ofthe partial image in the target direction may be calculated, the wrongidentification caused by the arm image with different numbers of fingersmay be avoided because if there is one less finger, at the fingerposition, the position with the most rapid change in slope may begenerated in the gray projection curve, thereby further improving theaccuracy of wrist identification.

As one optional embodiment, before the step of determining the targetposition in the binary image as the wrist position of the binary imageaccording to the gray projection curve, the method further includes:

calculating a second derivative of the gray projection curve, where theposition where the curve slope changes the most rapidly in the grayprojection curve is a position corresponding to a maximum value of thesecond derivative in the gray projection curve; or

calculating a first derivative of the gray projection curve, where theposition where the curve slope changes the most rapidly in the grayprojection curve is a position corresponding to a maximum value of thecurve slopes of the first derivative in the gray projection curve.

In the present embodiment, the projection position of the targetposition in the binary image in the target direction is the position ofthe maximum value in the second derivative, determined as the wristposition. For example, FIG. 6 shows the binary image, the partial image,the gray projection curve of the partial image, the first and secondderivatives of the gray projection curve, and the gesture image. In FIG.6, a two-dimensional coordinate system may be established. The ordinatesin the gray projection curve, the first derivative and the secondderivative represent numerical values of respective curves, and theabscissas represent that the gray projection curve, the first derivativeand the second derivative extend in the principal direction. In thisway, after the maximum value in the second derivative is determined, theabscissa value (such as the abscissa value shown by the imaginary lineas shown in FIG. 6) corresponding to the maximum value may bedetermined, thereby determining the target position of this abscissavalue in the binary image as the wrist position.

In addition, in the above-mentioned first derivative, the position withthe maximum value of the curve slope may be determined by calculatingthe curve slope of each position in the first derivative, therebydetermining the abscissa value (such as the abscissa value shown by theimaginary line) corresponding to the position with the maximum value ofthe curve slope, and then determining the target position of theabscissa value in the binary image as the wrist position.

In the present embodiment, since the wrist position may be determined bythe position of the maximum value of the second derivative or themaximum value of the curve slope in the first derivative, therebyimproving the accuracy of determining the wrist position, and theaccuracy of gesture identification.

For example, the human hand is characterized by extending from the armto the palm, from the arm to the wrist, with a little change in thewidth of the human hand, and an obvious change in the width at the wristposition. Correspondingly, in the first image, in the directionperpendicular to the principal direction, the position where the widthin the binary image changes the most violently is the wrist, therebyfinding the wrist position, and segmenting the gesture image and the armimage based on the wrist position. In the present embodiment, the wristposition may be determined by the method based on the projection curve.After the principal direction is determined, the binary image and thepartial image are rotated by an angle of θ (for example, the includedangle between the principal direction and horizontal direction), suchthat the principal direction of the gesture image becomes the horizontaldirection, and then the vertical direction of the rotated partial image,the gray projection curve, the first and second derivatives of the grayprojection curve are calculated respectively, as shown in FIG. 6. At thewrist position, the gray projection curve changes the most violently.That is, the slope of the gray projection curve changes the mostrapidly, i.e., the first derivative curve of the projection curvechanges the most rapidly. The position where the first derivative curvechanges the most rapidly is the position of the maximum value of theslope of the first derivative curve, which is the position of themaximum value of the second derivative curve of the projection curve.

In some optional embodiments, the maximum value is the maximum of thesecond derivatives within a search region, in which the search region isa remaining region of the x-coordinates obtained by removing a firstequal value region and a last equal value region from the secondderivative. The first equal value region ranges from the firstx-coordinate, the x-coordinates of the first equal value region arecontinuous, and the second derivatives under the continuousx-coordinates of the first equal value region have a same value equal toa value of the second derivative under the first x-coordinate. The lastequal value region ends up with the last x-coordinate, the x-coordinatesof the last equal value region are continuous, and the secondderivatives under the continuous x-coordinates of the last equal valueregion have a same value equal to a value of the second derivative underthe last x-coordinate.

The above-mentioned first and the last equal value regions may bereferred to as the second derivative regions corresponding to two zeropoints of the projection curve because the second derivative region atthe position of zero point of the projection curve is the equal valueregion, for example, the first line segment and the last line segment inthe second derivative parallel to a horizontal line as shown in FIG. 6.

In practical applications, the projection curve may change violently atthe two ends of the image. Therefore, at the two ends of the binaryimage, the gray projection value increases from the zero and finallyreduces to zero, thereby causing the maximum value point of the secondderivative of the projection curve to appear at the two ends of theimage. In this way, in the present embodiment, the maximum value is theone within the above-mentioned search region of the second derivative,which may avoid the maximum value of the second derivative to be at thetwo sides of the projection curve, so as to improve the accuracy ofdetermining the wrist position. Since in the normal first image, thewrist position is definitely located between two zero points, themaximum value of the second derivative of the projection curve issearched in a small region, to determine the wrist position, therebyimproving the accuracy of determining the wrist position.

As one optional embodiment, before the step of calculating the grayprojection curve of the partial image in the target direction, themethod further includes:

performing an opening operation on the binary image, at least removingthe finger information and noise part, so as to obtain the partialimage.

The above-mentioned opening operation is morphological, which includesan erosion operation on the binary image for removing information ofsmaller objects, and a dilation operation for restoring a shape of theremaining objects, so as to obtain the partial image from which at leastthe finger part and the noise part are removed. Specifically, referringto the change in FIGS. 2 and 7, FIG. 2 shows the binary image, and FIG.7 shows the partial image extracted from the binary image shown in FIG.2 by performing the opening operation. Certainly, the opening operationis performed on the above-mentioned binary image, to remove the fingerpart and the noise part, and image information of other small parts, fordetermining the principal direction and the wrist position subsequently.

In the present embodiment, the partial image is obtained by the openingoperation on the binary image to remove the finger part and the noisepart, thereby improving the accuracy of determining the principaldirection and the wrist direction from the partial image, and improvingthe accuracy of gesture identification.

It should be noted that in the present embodiment of the presentdisclosure, it is not limited that the above-mentioned partial image isobtained by the opening operation. For example, the finger part of thebinary image may be removed by a Hough circle detection method or aneural network approach, so as to obtain the above-mentioned partialimage.

As one of the optional embodiments, the step of identifying theprincipal direction of the partial image includes:

detecting N line segments of a contour of the partial image, andcalculating a length of each of the N line segments and an includedangle between each of the N line segments and a reference direction,where N is a positive integer, and the N line segments represent thecontour of the hand and the wrist;

determining a target angle according to the lengths of the N linesegments, and the included angles between the N line segments and thereference direction; and

determining the principal direction of the partial image according tothe target angle, the target angle being an included angle between theprincipal direction and the reference direction.

In the step of detecting N line segments of a contour of the partialimage, the contour of the partial image is determined by the gray valueof each pixel point in the partial image, because the arm part and thehand part each has a gray value of 255, and the other part in partialimage has a gray value of 0. Afterwards, N line segments of the contourof the above-mentioned partial image may be detected by the Houghtransform line segment detection method or the Line Segment Detector(LSD) method. The above-mentioned N line segments may be seven linesegments as shown in FIG. 8. Certainly, there may be more line segments.The larger the number of line segments, the closer the calculatedprincipal direction of the binary image to reality, and the greater thecomputational load. In practical applications, the number of linesegments may be set reasonably as needed.

The above-mentioned preset reference direction may be the horizontaldirection or the vertical direction, or other directions, which is notlimited in the present disclosure. In the drawings in the embodiments ofthe present disclosure, the preset reference direction is taken as thehorizontal direction.

The step of determining a target angle according to the lengths of the Nline segments, and the included angles between the N line segments andthe reference direction may include:

calculating the target angle through dividing a sum of weighted includedangles of the N line segments by a sum of lengths of the N linesegments, in which the weighted included angle of the i-th line segmentis a product of the included angle between the i-th line segment and thereference direction, and the length of the i-th line segment, and i isany integer ranging from 1 to N.

The sum of the weighted included angles may be that of the N linesegments. Since the weighted included angle of the i-th line segment isthe product obtained by multiplying the included angle between the linesegment and the preset reference direction by the length of the i-thline segment, thereby taking the length of each line segment as aweighted value of this line segment. Since in practice, the part with agreater length has a greater influence on the principal direction, theaccuracy of the principal direction may be improved.

For example, as shown in FIG. 8, in the embodiment of the presentdisclosure, in view of the structural characteristics of the human hand,the partial image is symmetrical about the principal directionapproximately. Therefore, the principal direction of the first image isdetermined according to the length and the direction of each linesegment in the image, and the principal direction of the first image issimilar to the direction of the longest line segment (for example, whenthe arm in the image is relatively big, the principal direction of theimage is similar to the direction of the two line segments of the armedge, and the line segments 1 and 4 are line segments of the arm edge).That is, for the relatively longer line segment, its direction (forexample, the included angle with the horizontal direction) has arelatively large influence on the principal direction of the firstimage. Therefore, in the embodiment of the present disclosure, theprincipal direction of the first image may be determined using theweighted average method, in which the length of each line segment istaken as a weighting coefficient. The longer the line segment, thegreater the weighting coefficient corresponding to its direction. TakingFIG. 8 as an example, there are mainly 7 line segments. It is easy tocalculate the lengths of the 7 line segments l₁, l₂, l₃, . . . l₇, andthe included angles θ₁, θ₂, θ₃, . . . θ₇ between the 7 line segments andthe horizontal direction. By taking the length of each line segment asthe weighting coefficient, the weighted average result of the directionsof the 7 line segments is calculated, thereby determining the principaldirection of the arm and the gesture image with the following formula:

$\theta = {\frac{\sum\limits_{i = 1}^{7}{l_{i} \times \theta_{i}}}{\sum\limits_{i = 1}^{7}l_{i}}.}$

In the above-mentioned formula, the length l_(i) of each line segment istaken as the weighting coefficient of its direction (for example, theincluded angle θ_(i) with the horizontal direction). The numerator is aweighted sum of directions of the seven line segments (for example, theincluded angle with the horizontal direction), and the denominator isthe sum of seven weighting coefficients, taking the final weightedaverage result θ as the included angle between the principal directionand the preset reference direction (such as the horizontal direction).

Certainly, in the embodiment of the present disclosure, the step ofdetermining the target angle according to the lengths of the N linesegments and the included angles between the N line segments and thepreset reference direction is not limited to the method of dividing asum of weighted included angles of the N line segments by a sum oflengths of the N line segments. For example, an average value of theincluded angles between the two longest line segments of the N linesegments and the preset reference direction may also be taken as theabove-mentioned target angle, which is not limited herein.

In the embodiment shown in FIG. 4, the wrist position of the binaryimage is determined by the position where the slope changes the mostrapidly in the gray projection curve, thereby accurately determining thewrist position, and further improving the accuracy of gestureidentification.

FIG. 9 is a flow chart of a gesture identification method according toat least one embodiment of the present disclosure. As shown in FIG. 9,the method includes the following steps 901 to 904.

Step 901: obtaining a first image.

Step 902: determining a wrist position using the wrist identificationmethod according to the embodiment of the present disclosure.

Step 903: segmenting the binary image along the wrist position, toobtain a gesture image.

Step 904: performing gesture identification on the gesture image.

It should be noted that the step 902 of determining the wrist positionmay refer to the embodiments shown in FIGS. 1 to 4, which is notrepeated herein.

In the present embodiment, after the wrist position is determined, thebinary image is segmented, to obtain the gesture image, therebyperforming gesture identification on this gesture image, for example,identifying a trajectory of change or size or space position of thegesture image, which is not limited herein.

The binary image is segmented along the wrist position, to obtain thegesture image, which may prevent the arm image from interfering with thegesture identification, improving the accuracy of gestureidentification.

It should be noted that the gesture identification method according tothe embodiment of the present disclosure may be applied to anyelectronic equipment capable of identifying images, such as a mobilephone, a tablet PC, a computer, a game device, an intelligent controldevice or a TV, which is not limited herein.

In the present embodiment, since the wrist position may be determinedaccurately, and the binary image is segmented along the wrist position,and the arm part may be removed, thereby avoiding the influence of thearm part on the gesture identification, so as to improve the accuracy ofgesture identification.

FIG. 10 is a structural diagram of a wrist identification deviceaccording to at least one embodiment of the present disclosure. As shownin FIG. 10, the wrist identification device 1000 includes:

an obtaining module 1001, configured to obtain a first image, the firstimage including a hand and a wrist;

a first processing module 1002, configured to binarize the first imageto obtain a binary image, and extract a partial image from the binaryimage, where the partial image is obtained by removing at least offinger information from the binary image;

an identifying module 1003, configured to identify a principal directionof the binary image based on the partial image, and determine a targetdirection perpendicular to the principal direction; and

a determining module 1004, configured to determine a target position inthe binary image where the binary image matches a first wrist feature asa wrist position in the binary image according to the target direction.

In some optional embodiments, as shown in FIG. 11, the determiningmodule 1004 includes:

a first calculating unit 10041, configured to calculate a grayprojection curve of the binary image or a gray projection curve of thepartial image in the target direction; and

a determining unit 10042, configured to determine the target position inthe binary image as the wrist position in the binary image according tothe gray projection curve of the binary image or the gray projectioncurve of the partial image, where a projection position of the targetposition in the target direction is a position where a curve slopechanges the fastest in the gray projection curve.

In some optional embodiments, as shown in FIG. 12, the device furtherincludes:

a second calculating unit 1005, configured to calculate a secondderivative of the gray projection curve, where the position where thecurve slope changes the fastest in the gray projection curve is aposition corresponding to a maximum value of the second derivative inthe gray projection curve; or

a third calculating unit 1006, configured to determine the targetposition in the binary image as the wrist position in the binary imageaccording to the gray projection curve of the binary image or the grayprojection curve of the partial image, where a projection position ofthe target position in the target direction is a position where a curveslope changes the fastest in the gray projection curve.

In some optional embodiments, as shown in FIG. 13, the device furtherincludes:

a second processing module 1007, configured to perform an openingoperation on the binary image to remove at least of the fingerinformation and a noise, so as to obtain the partial image.

In some optional embodiments, as shown in FIG. 14, the identifyingmodule 1003 includes:

a detecting unit 10031, configured to detect N line segments of acontour of the partial image, and calculate a length of each of the Nline segments and an included angle between each of the N line segmentsand a reference direction, where N is a positive integer, and the N linesegments represent the contour of the hand and the wrist;

a first determining unit 10032, configured to determine a target angleaccording to the lengths of the N line segments, and the included anglesbetween the N line segments and the reference direction; and

a second determining unit 10033, configured to determine the principaldirection of the partial image according to the target angle, the targetangle being an included angle between the principal direction and thereference direction.

In some optional embodiments, the first determining unit 10032 isconfigured to take the angle obtained by dividing a sum of weightedincluded angles of the N line segments by a sum of lengths of the N linesegments as the target angle, in which the weighted included angle ofthe i-th line segment is a product obtained by multiplying the includedangle between the i-th line segment and the preset reference directionby the length of the i-th line segment, and i is any one integer from 1to N.

In some optional embodiments, the principal direction is a row directionof the binary image, the target direction is a column of the binaryimage, x-coordinates of the gray projection curve are continuous columnnumbers of the binary image, and a y-coordinate of the gray projectioncurve under any one of the x-coordinates is a sum of gray values of allpixels of in the corresponding column of the binary image. The maximumvalue is the maximum value of the second derivatives of the grayprojection curve within a search region, and the search region is aremaining region of the x-coordinates obtained by removing a first equalvalue region and a last equal value region from the second derivative.The first equal value region ranges from the first x-coordinate, thex-coordinates of the first equal value region are continuous, and thesecond derivatives under the continuous x-coordinates of the first equalvalue region have a same value equal to a value of the secondderivatives under the first x-coordinate. The last equal value regionends up with the last x-coordinate, the x-coordinates of the last equalvalue region are continuous, and the second derivatives under thecontinuous x-coordinates of the last equal value region have a samevalue equal to a value of the second derivative under the lastx-coordinate.

It should be noted that the above-mentioned wrist identification device1000 in the present embodiment may implement the wrist identificationmethod in any embodiment of the present disclosure. That is, the wristidentification method according to any embodiment of the presentdisclosure may be implemented by the above-mentioned wristidentification device 1000 in the present embodiment, and achieve thesame advantageous effects, which is not repeated herein.

FIG. 15 is a structural diagram of a gesture identification deviceaccording to at least one embodiment of the present disclosure. As shownin FIG. 15, the gesture identification device includes:

an obtaining module 1501, configured to obtain a first image, the firstimage including a hand and a wrist;

a determining module 1502, configured to determine the wrist positionusing the wrist identification according to the embodiment of thepresent disclosure;

a segmenting module 1503, configured to segment the binary image alongthe wrist position, to obtain a gesture image; and

an identifying module 1504, configured to perform gesture identificationon the gesture image.

It should be noted that the above-mentioned gesture identificationdevice 1500 in the present embodiment may implement the gestureidentification method in any embodiment of the present disclosure. Thatis, the gesture identification method according to any embodiment of thepresent disclosure may be implemented by the above-mentioned gestureidentification device 1500 in the present embodiment, and achieve thesame advantageous effects, which is not repeated herein.

Reference is made to FIG. 16, which is a structural schematic diagram ofelectronic equipment according to at least one embodiment of the presentdisclosure. The electronic equipment 1600 includes: a memory 1601, aprocessor 1602 and a computer program which is stored on the memory 1601and executable by the processor 1602. The processor 1602 is configuredto read the computer program in the memory 1601 and perform thefollowing processes:

obtaining a first image, the first image including a hand and the wrist;

binarizing the first image to obtain a binary image;

extracting a partial image from the binary image, where the partialimage is obtained by removing at least of finger information from thebinary image;

identifying a principal direction of the binary image based on thepartial image, and determining a target direction perpendicular to theprincipal direction; and

determining a target position in the binary image where the binary imagematches a first wrist feature as a wrist position in the binary imageaccording to the target direction.

In some embodiments, in determining a target position in the binaryimage where the binary image matches a first wrist feature as a wristposition in the binary image according to the target direction, theprocessor 1602 is further configured to:

calculate a gray projection curve of the binary image or a grayprojection curve of the partial image in the target direction; and

determine the target position in the binary image as the wrist positionin the binary image according to the gray projection curve of the binaryimage or the gray projection curve of the partial image, where aprojection position of the target position in the target direction is aposition where a curve slope changes the fastest in the gray projectioncurve.

In some optional embodiments, prior to the determining a target positionin the binary image where the binary image matches a first wrist featureas a wrist position in the binary image according to the targetdirection, the processor 1602 is further configured to:

calculate a second derivative of the gray projection curve, where theposition where the curve slope changes the fastest in the grayprojection curve is a position corresponding to a maximum value of thesecond derivative in the gray projection curve; or

calculate a first derivative of the gray projection curve, where theposition where the curve slope changes the fastest in the grayprojection curve is a position corresponding to a maximum value of thecurve slopes of the first derivative in the gray projection curve.

In some optional embodiments, prior to the calculating a gray projectioncurve of the partial image in the target direction, the processor 1602is further configured to:

perform an opening operation on the binary image to remove at least ofthe finger information and a noise, so as to obtain the partial image.

In some optional embodiments, in identifying a principal direction ofthe binary image based on the partial image, the processor 1602 isfurther configured to:

detect N line segments of a contour of the partial image, andcalculating a length of each of the N line segments and an includedangle between each of the N line segments and a reference direction,where N is a positive integer, and the N line segments represent thecontour of the hand and the wrist;

determine a target angle according to the lengths of the N linesegments, and the included angles between the N line segments and thereference direction; and

determine the principal direction of the partial image according to thetarget angle, the target angle being an included angle between theprincipal direction and the reference direction.

In some optional embodiments, in determining a target angle according tothe lengths of the N line segments, and the included angles between theN line segments and the reference direction, the processor 1602 isfurther configured to:

calculate the target angle through dividing a sum of weighted includedangles of the N line segments by a sum of lengths of the N linesegments, in which the weighted included angle of the i-th line segmentis a product of the included angle between the i-th line segment and thereference direction, and the length of the i-th line segment, and i isany integer ranging from 1 to N.

In some optional embodiments, the principal direction is a row directionof the binary image, the target direction is a column of the binaryimage, x-coordinates of the gray projection curve are continuous columnnumbers of the binary image, and a y-coordinate of the gray projectioncurve under any one of the x-coordinates is a sum of gray values of allpixels of in the corresponding column of the binary image. The maximumvalue is the maximum value of the second derivatives of the grayprojection curve within a search region, and the search region is aremaining region of the x-coordinates obtained by removing a first equalvalue region and a last equal value region from the second derivative.The first equal value region ranges from the first x-coordinate, thex-coordinates of the first equal value region are continuous, and thesecond derivatives under the continuous x-coordinates of the first equalvalue region have a same value equal to a value of the secondderivatives under the first x-coordinate. The last equal value regionends up with the last x-coordinate, the x-coordinates of the last equalvalue region are continuous, and the second derivatives under thecontinuous x-coordinates of the last equal value region have a samevalue equal to a value of the second derivative under the lastx-coordinate.

It should be noted that the above-mentioned electronic equipment 1600 inthe present embodiment may implement the wrist identification method inany embodiment of the present disclosure. That is, the wristidentification method according to any embodiment of the presentdisclosure may be implemented by the above-mentioned electronicequipment 1600 in the present embodiment, and achieve the sameadvantageous effects, which is not repeated herein.

Reference is made to FIG. 17, which is a structural schematic diagram ofelectronic equipment according to at least one embodiment of the presentdisclosure. The electronic equipment 1700 includes: a memory 1701, aprocessor 1702 and a computer program which is stored on the memory 1701and executable by the processor 1702. The processor 1702 is configuredto read the computer program in a memory 1701 to execute the followingprocesses:

obtaining a first image, the first image including a hand and a wrist;

determining the wrist position using the wrist identification accordingto the embodiments of the present disclosure;

segmenting the binary image along the wrist position, to obtain agesture image; and

performing gesture identification on the gesture image.

It should be noted that the above-mentioned electronic equipment 1700 inthe present embodiment may implement the gesture identification methodin any embodiment of the present disclosure. That is, the gestureidentification method according to any embodiment of the presentdisclosure may be implemented by the above-mentioned electronicequipment 1700 in the present embodiment, and achieve the sameadvantageous effects, which is not repeated herein.

At least one embodiment of the present disclosure further provides acomputer-readable storage medium storing a computer program. When thecomputer program is executed by the processor, steps in the wristidentification method or the gesture identification method according tothe embodiments of the present disclosure are implemented.

It should be noted that, various units or modules included in thedevices of the above embodiments are divided only according to logicalfunctions, but not limited to the above division manner, as long as itcan implement corresponding functions. In addition, names of variousfunctional units or modules are only for ease of distinction, but not tolimit the protection scope of the present disclosure.

It may be understood by those skilled in the art, all or part of stepsimplementing the above-described embodiments may be completed byhardware instructed by relevant program instructions, which can bestored in a computer readable storage medium. The above-mentionedstorage medium may be a read-only memory, a disk or a CD etc.

The above descriptions are merely optional embodiments of the presentdisclosure. It should be noted that several improvements andmodifications may be made for a person skilled in the art withoutdeparting from the principle of the present disclosure, and also shouldbe considered to fall within the protection scope of the presentdisclosure.

What is claimed is:
 1. A method for identifying a wrist, comprising:obtaining a first image, the first image comprising a hand and thewrist; binarizing the first image to obtain a binary image; extracting apartial image from the binary image, wherein the partial image isobtained by removing at least of finger information from the binaryimage; identifying a principal direction of the binary image based onthe partial image, and determining a target direction perpendicular tothe principal direction; and determining a target position in the binaryimage where the binary image matches a first wrist feature as a wristposition in the binary image according to the target direction.
 2. Themethod according to claim 1, wherein the determining a target positionin the binary image where the binary image matches a first wrist featureas a wrist position in the binary image according to the targetdirection comprises: calculating a gray projection curve of the binaryimage or a gray projection curve of the partial image in the targetdirection; and determining the target position in the binary image asthe wrist position in the binary image according to the gray projectioncurve of the binary image or the gray projection curve of the partialimage, wherein a projection position of the target position in thetarget direction is a position where a curve slope changes the fastestin the gray projection curve.
 3. The method according to claim 2,wherein prior to the determining a target position in the binary imagewhere the binary image matches a first wrist feature as a wrist positionin the binary image according to the target direction, the methodfurther comprises: calculating a second derivative of the grayprojection curve, wherein the position where the curve slope changes thefastest in the gray projection curve is a position corresponding to amaximum value of the second derivative in the gray projection curve; orcalculating a first derivative of the gray projection curve, wherein theposition where the curve slope changes the fastest in the grayprojection curve is a position corresponding to a maximum value of thecurve slopes of the first derivative in the gray projection curve. 4.The method according to claim 2, wherein prior to the calculating a grayprojection curve of the partial image in the target direction, themethod further comprises: performing an opening operation on the binaryimage to remove at least of the finger information and a noise, so as toobtain the partial image.
 5. The method according to claim 3, whereinprior to the calculating a gray projection curve of the partial image inthe target direction, the method further comprises: performing anopening operation on the binary image to remove at least of the fingerinformation and a noise, so as to obtain the partial image.
 6. Themethod according to claim 4, wherein the identifying a principaldirection of the binary image based on the partial image comprises:detecting N line segments of a contour of the partial image, andcalculating a length of each of the N line segments and an includedangle between each of the N line segments and a reference direction,wherein N is a positive integer, and the N line segments represent thecontour of the hand and the wrist; determining a target angle accordingto the lengths of the N line segments, and the included angles betweenthe N line segments and the reference direction; and determining theprincipal direction of the partial image according to the target angle,the target angle being an included angle between the principal directionand the reference direction.
 7. The method according to claim 6, whereinthe determining a target angle according to the lengths of the N linesegments, and the included angles between the N line segments and thereference direction comprises: calculating the target angle throughdividing a sum of weighted included angles of the N line segments by asum of lengths of the N line segments, in which the weighted includedangle of the i-th line segment is a product of the included anglebetween the i-th line segment and the reference direction, and thelength of the i-th line segment, and i is any integer ranging from 1 toN.
 8. The method according to claim 6, wherein the determining a targetangle according to the lengths of the N line segments, and the includedangles between the N line segments and the reference directioncomprises: determining the target angle θ according to a formula asfollowing:$\theta = \frac{\sum\limits_{i = 1}^{7}{l_{i} \times \theta_{i}}}{\sum\limits_{i = 1}^{7}l_{i}}$where l_(i) is the length of the i-th line segment, θ_(i) is theincluded angle between the i-th line segment and the referencedirection, and 1≤i≤N.
 9. The method according to claim 3, wherein theprincipal direction is a row direction of the binary image, the targetdirection is a column of the binary image, x-coordinates of the grayprojection curve are continuous column numbers of the binary image, anda y-coordinate of the gray projection curve under any one of thex-coordinates is a sum of gray values of all pixels of in thecorresponding column of the binary image, wherein the maximum value isthe maximum value of the second derivatives of the gray projection curvewithin a search region, and the search region is a remaining region ofthe x-coordinates obtained by removing a first equal value region and alast equal value region from the second derivative, and wherein thefirst equal value region ranges from the first x-coordinate, thex-coordinates of the first equal value region are continuous, and thesecond derivatives under the continuous x-coordinates of the first equalvalue region have a same value equal to a value of the second derivativeunder the first x-coordinate, and wherein the last equal value regionends up with the last x-coordinate, the x-coordinates of the last equalvalue region are continuous, and the second derivatives under thecontinuous x-coordinates of the last equal value region have a samevalue equal to a value of the second derivative under the lastx-coordinate.
 10. A method for identifying a gesture, comprising:determining the wrist position using the method for identifying a wristaccording to claim 1; segmenting the binary image along the wristposition to obtain a gesture image; and identifying the gesture on thegesture image.
 11. A device for identifying a gesture, comprising amemory, a processor and a program that is stored on the memory andexecutable by the processor, wherein when the program is executed by theprocessor, the processor is configured to: obtain a first image, thefirst image comprising a hand and a wrist; binarize the first image toobtain a binary image; extract a partial image from the binary image,wherein the partial image is obtained by removing at least of fingerinformation from the binary image; identify a principal direction of thebinary image based on the partial image, and determine a targetdirection perpendicular to the principal direction; and determine atarget position in the binary image where the binary image matches afirst wrist feature as a wrist position in the binary image according tothe target direction.
 12. The device according to claim 11, wherein indetermining a target position in the binary image where the binary imagematches a first wrist feature as a wrist position in the binary imageaccording to the target direction, the processor is further configuredto: calculate a gray projection curve of the binary image or a grayprojection curve of the partial image in the target direction; anddetermine the target position in the binary image as the wrist positionin the binary image according to the gray projection curve of the binaryimage or the gray projection curve of the partial image, wherein aprojection position of the target position in the target direction is aposition where a curve slope changes the fastest in the gray projectioncurve.
 13. The device according to claim 12, wherein, prior to thedetermining a target position in the binary image where the binary imagematches a first wrist feature as a wrist position in the binary imageaccording to the target direction, the processor is further configuredto: calculate a second derivative of the gray projection curve, whereinthe position where the curve slope changes the fastest in the grayprojection curve is a position corresponding to a maximum value of thesecond derivative in the gray projection curve; or calculate a firstderivative of the gray projection curve, wherein the position where thecurve slope changes the fastest in the gray projection curve is aposition corresponding to a maximum value of the curve slope of thefirst derivative in the gray projection curve.
 14. The device accordingto claim 13, wherein in identifying a principal direction of the binaryimage based on the partial image, the processor is further configuredto: detect N line segments of a contour of the partial image, andcalculate a length of each of the N line segments and an included anglebetween each of the N line segments and a reference direction, wherein Nis a positive integer, and the N line segments represent the contour ofthe hand and the wrist; determine a target angle according to thelengths of the N line segments, and the included angles between the Nline segments and the reference direction; and determine the principaldirection of the partial image according to the target angle, the targetangle being an included angle between the principal direction and thereference direction.
 15. The device according to claim 14, wherein indetermining a target angle according to the lengths of the N linesegments, and the included angles between the N line segments and thereference direction, the processor is further configured to determinethe target angle θ according to a formula as following:$\theta = \frac{\sum\limits_{i = 1}^{7}{l_{i} \times \theta_{i}}}{\sum\limits_{i = 1}^{7}l_{i}}$where l_(i) is the length of the i-th line segment, θ_(i) is theincluded angle between the i-th line segment and the referencedirection, and 1≤i≤N.
 16. The device according to claim 13, wherein theprincipal direction is a row direction of the binary image, the targetdirection is a column of the binary image, x-coordinates of the grayprojection curve are continuous column numbers of the binary image, anda y-coordinate of the gray projection curve under any one of thex-coordinates are a sum of gray values of all pixels of in thecorresponding column of the binary image, wherein the maximum value isthe maximum value of the second derivatives of the gray projection curvewithin a search region, and the search region is a remaining region ofthe x-coordinates obtained by removing a first equal value region and alast equal value region from the second derivative, and wherein thefirst equal value region ranges from the first x-coordinate, thex-coordinates of the first equal value region are continuous, and thesecond derivatives under the continuous x-coordinates of the first equalvalue region have a same value equal to a value of the secondderivatives under the first x-coordinate, and wherein the last equalvalue region ends up with the last x-coordinate, the x-coordinates ofthe last equal value region are continuous, and the second derivativesunder the continuous x-coordinates of the last equal value region have asame value equal to a value of the second derivatives under the lastx-coordinate.
 17. Electronic equipment, comprising a memory, a processorand a program that is stored on the memory and executable by theprocessor; wherein when the program is executed by the processor, theprocessor is configured to: determine the wrist position using themethod for identifying a wrist according to claim 1; segment the binaryimage along the wrist position, to obtain a gesture image; and identifythe gesture on the gesture image.
 18. A computer-readable storagemedium, storing a computer program, wherein when the computer program isexecuted by a processor, the method for identifying a wrist according toclaim 1 is implemented.
 19. A computer-readable storage medium, storinga computer program, wherein when the computer program is executed by aprocessor, the method for identifying a gesture according to claim 10 isimplemented.